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The  transition  from  static  to  motion  imagery  enabled  by  UAVs,  and  distributed  sensor 
networks  introduces  significant  challenges  -  monitoring  multiple  video  streams,  indexing 
large  amounts  of  video,  and  retrieving  video  segments  with  significant  spatiotemporal 
events.  Current  analyst  workflow  systems  support  spatiotemporal  analysis  minimally: 
analysts  roam  through  large  image  mosaics,  revisit  the  same  area,  and  make  use  of 
accompanying  geographic  information  systems  (GIS)  databases  to  examine  the  evolution 
of  a  scene.  However,  the  analyst  still  needs  to  decide  whether  the  trajectory  of  a  vehicle 
appears  suspicious,  whether  a  moving  convoy  resembles  a  formation  of  military  vehicles, 
or  whether  the  trajectory  of  a  truck  today  resembles  yesterday’s  pattern  of  movement  of 
another  vehicle.  As  data  influx  is  increasing  at  substantially  high  rates,  agencies  are 
trying  hard  to  keep  up  with  tremendous  amounts  of  incoming  data,  further  exasperated 
during  crises  (e.g.  military  operations),  where  the  rate  of  incoming  information  explode 
(e.g.  as  areas  of  interest  are  under  heavy  surveillance  by  fleets  of  UAVs). 

What  is  needed  is  a  geospatial  exploitation  of  motion  imagery  (GEMI)  system  that 
automates  monitoring,  indexing,  search  and  retrieval  at  a  level  of  abstraction  supporting 
the  analyst  workflow.  Such  a  system  should  enable  an  analyst  to  query  motion  trajectory 
events  such  as: 

■  Monitor  or  search  for  video  segments  containing  traffic  similar  to  a  specified 
pattern  (e.g.  a  vehicle  coming  to  a  stop  on  the  side  of  the  highway,  human  activity 
near  the  stopped  vehicle,  and  vehicle  taking  off  after  the  stop) 

■  Monitor  or  search  for  video  segments  containing  motion  near  a  given  highway 
(e.g.  human  activity  in  fields  near  the  highway) 

■  Monitor  or  search  for  video  segments  containing  unusual  traffic  activity  deviating 
from  the  norm  (e.g.  deviations  from  temporal  patterns,  deviations  from  spatial 
patterns,  etc) 

■  Monitor  or  search  for  video  segments  containing  specific  group  patterns  (e.g. 
crowd  activity  moving  towards  a  convoy  of  vehicles) 

In  this  paper,  we  present  a  motion  pattern  modeling,  identification  and  indexing  method 
based  on  our  innovative  model  of  spatiotemporal  helixes  (Agouris  &  Stefanidis,  2003)1, 
and  feasibility  demonstration  results  for  application  to  urban  scenes  using  video  imagery. 
Our  research  has  shown  the  feasibility  of  modeling  video  motion  trajectories  with 
spatiotemporal  helixes,  and  using  this  model  as  a  monitoring  and  indexing  mechanism  for 
storage  and  query  of  spatiotemporal  events.  Our  model  captures  the  point-mass  motion 
trajectory  with  an  optimal  number  of  nodes  using  self-optimizing  neural  networks. 
Similarly,  our  threshold-based  approach  captures  motion  dynamics  (expansion  or  collapse 
of  the  object’s  outline)  with  an  optimal  set  of  prongs. 
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In  order  to  compare  helix  motion  models,  we  have  developed  similarity  metrics  to 
compare  helixes  using  their  node  and  prong  information.  Two  helixes  would  be 
considered  identical  when  their  nodes  and  prongs  follow  similar  patterns  (displaying 
similar  attributes  at  similar  relative  instances).  Variations  in  these  attributes  would  reduce 
the  similarity  of  two  helixes.  The  similarity  metrics  developed  enable  the  comparison  of 
trajectory  models  -  at  the  coarse  level  -  with  mobility  state  transition  metrics,  and  -  at  a 
granular  level  -  with  spatiotemporal  distance  metrics.  Mobility  state  transition  metrics 
perform  abstract  comparisons  based  on  the  high  level  attributes  of  node  accelerations  and 
decelerations;  prong  rotations,  contractions  and  expansions.  Similarity  metrics  compute 
the  Euclidean  distance  for  the  same  set  of  comparison  variables. 

The  accompanying  figure  on  the  left  shows  the  helix  for  an 
individual  monitored  by  a  UAV  overlaid  onto  a  virtual 
mosaic.  The  polygonic  line  linking  7  nodes  (white  boxes) 
depicts  the  helix,  and  the  red  box  indicates  the  last  tracked 
record  of  the  person.  The  helix  comprises  only  a  spine  in 
this  case,  as  the  person  is  represented  as  a  point  mass. 

Prongs  depicted  in  the  accompanying  figure  on  the  right 
capture  the  expansion  or  contraction  of  the  object  (e.g.  a 
convoy  of  vehicles,  crowd  demonstrating  at  a  city  square) 
Arrows  pointing  out  of 
the  helix  spine  denote 

expansion  whereas  arrows  into  the  spine  depict 
contraction  of  object  outline. 

In  experiments  with  the  spatiotemporal  similarity 
metrics  developed,  distorting  a  helix  through  rotation 
and/or  scaling  has  a  nearly  linear  effect  on  the 
resulting  quantitative  similarity  metrics  (distance 
metrics),  while  abstract  metrics  (mobility  state 
transition)  behave  very  robustly,  remaining  almost 
unaffected  as  long  as  the  distortions  leave  the 
nodes/prongs  within  the  corresponding  search 
windows. 


In  the  full  paper,  we  will  present  detailed  empirical  results  obtained  with  the  UAV  data 
showing  the  representation  efficiency  and  retrieval  performance  of  our  GEMI  system. 

Our  R&D  will  enable  the  DoD  image  analysts  query  motion  trajectory  events  in  multiple 
video  streams  or  databases  while  using  video  registration  software  to  integrate  our  query 
responses  into  virtual  mosaics. 
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